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Higher education campus leaders face a complex state of affairs regarding the documentation of evidence of 
student learning. There is no shortage of technical guidance for conducting assessment (e.g. Allen, 2006; 
Bresciani, 2007; Bresciani, Zelna, & Anderson, 2004; Lui, 2011; Maki, 2010; Suskie, 2009; Walvrood & 
Anderson, 2010), and a great deal of energy and resources are expended gathering, analyzing, interpreting, 
disseminating, and using data generated through this methodological advice. Yet, the advancement of 
assessment methods has outpaced explorations of assessment’s philosophy and discourses of how assessment 
and campus cultures are changed have been slow to emerge. In essence, the art and science of assessment are 
divided and, as Snow (1959) cautions, “when those two senses have grown apart, then no society is going to be 
able to think with wisdom ” (p. 29). As higher education places greater emphasis on empirical data from 
standardized learning, research regarding why assessment is conducted, how it is leveraged for change, and the 
ramifications of assessment’s purposes must be elevated to a more meaningful level. To this end a new 
instrument—The Survey of Assessment Culture ®— was developed to explore factors and strategies influencing 
the cultivation of cultures of assessment. The Survey supports research and dialogue into cultures of assessment 
and how assessment emerges as an accepted institutional way of existence. This article reviews the 
methodological approaches used in the study, shares basic descriptive statistics, and concludes by discussing 
various implications for the study of assessment cultures and for administrative practice in higher education and 
educational administration preparation programs. 


INTRODUCTION AND RESEARCH GOALS 

Popularly theorized by noted assessment scholar, Trudy Banta (2002), a culture of assessment 
refers to the deeply embedded values and beliefs collectively held by members of an 
institution influencing assessment practices on their campus (Banta & Associates, 2002; 
Banta, Lund, Black, Oblander, 1996). A culture of assessment is the primary and often 
unexplored system undergirding assessment practice on a campus. It is the system of thought 
and action reinforcing what “good” conduct of assessment looks like on a campus. Extending 
the concept of a culture of assessment further, Maki’s (2010) Principles of an Inclusive 
Commitment offer a structure of institutional partnerships, which, when operating efficiently, 
indicate a commitment to assessment of student learning. Maki (2010, p. 9) writes: 
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An inclusive commitment to assessment of student learning is established when it is 
(1) meaningfully anchored in the educational values of an institution—articulated in a 
principles-of-commitment statement; (2) intentionally designed to foster interrelated 
positions of inquiry about the efficacy of education practices among educators, 
students, and the institution itself as a learning organization; and (3) woven into roles 
and responsibilities across an institution from the chief executive officer through 
senior administrators, faculty leaders, faculty, staff, and students, (p. 3) 

Drawing from Maki’s work, a culture of assessment is defined (in this research endeavor) as 
the overarching ethos that is both an artifact of the way in which assessment is done and 
simultaneously a factor influencing and augmenting assessmen t practice. 

Guided by this scholarship, the Survey explores six constructs: a) Shared Institutional 
Commitment, b) Clear Conceptual Framework for Assessment, c) A Cross Institutional 
Responsibility, d) Transparency of Findings, e) Connection to Change-Making Processes, and 
f) Recognition of Leadership or Involvement in Assessment. The research goals for this long- 
range study include the exploration of factors supporting or impeding institutional capacities 
to develop, maintain, or augment a culture of assessment on their campus. In support of this 
goal, this study offers a description of the responses to various practices and tactics used by 
assessment practitioners across the United States. 

MATERIALS AND METHODS 

The Survey of Assessment Culture is a twenty-one item, electronic survey that collects forty- 
six data points divided into five parts: a) Identifying respondent’s role/ Chief Assessment 
Officer, b) Purpose for assessment, c) Assessment Culture Scales, d) Support, resistance, or 
indifference rankings, and e) Consent to follow up studies/contact. In 2011, the Survey was 
administered to a representative sample of U.S. institutional research and assessment directors 
to determine the usefulness and consistency of questions and generate information for 
instrument improvement. In general, the instrument was well developed and required 
minimal revision. For example, internal consistency measures (Cronbach’s alpha) for the 
Assessment Culture Scale items were 0.922, well above the generally-accepted 0.7 threshold. 

Sampling Method and Administration 

Prior research has relied on samples of convenience to explore assessment practices. This 
study relies on publically-available resources to construct a stratified, representative sample of 
the U.S. directors of institutional research and assessment. A listing of undergraduate, 
degree-granting, regionally-accredited institutions was downloaded from the Carnegie 
Classification of Institutions of Higher Education website 1 and was stratified according to 
institutional full-time enrollment size, accreditation region, and Carnegie Basic Classification. 
This stratified listing of institutions was placed in a sampling matrix according to the type of 
degrees awarded (primarily associates vs. primarily bachelors), regional accreditation region, 
and size of full-time enrollment [Small (under 1,999), Medium (2001 to 4,999), Large (5000 
to 9,999), and Very Large (Over 10,000)]. This resulted in a listing of 2,617 institutions; a 
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population similar to those surveyed by Kuh and Ikenberry (2009). Institutions were sampled 
at the most refined level of stratification and were over-sampled by a factor of three to ensure 
the best possible dispersion of a representative number of respondents at and across each level 
of stratification. 

A total of 1,026 institutions were randomly sampled for invitation to participate in the 
Survey. The Higher Education Directory 0 R was utilized to identify the contact information 
for directors of institutional research and assessment at sampled institutions. Although the 
Higher Education Directory is a voluntary listing of contact information, 77.2% or 792 email 
addresses for contacts were obtained using this resource. The remaining institutional contacts 
underwent status checks using institutional websites and public search engines 2 to identify 
Chief Assessment Officers; the individuals for whom assessment is their primary 
responsibility. One hundred and seventy Chief Assessment Officers were identified using this 
method. The remaining 64 participants did not have an entry in the Higher Education 
Directory and web searches did not yield contact information. In these cases, the Provost of 
the institution was invited to participate in the survey and his/her contact information was 
gathered using the Higher Education Directory. [Once the 1,026 survey respondents were 
invited to participate in the Survey, a total of 109 emails were returned as either inaccurate or 
no longer active. It can be assumed a total of 917 participants were adequately invited to 
participate in the survey]. A complete overview of methodology and limitations can be found 
at http://www.shsu.edu/assessmentculture/ 

RESULTS 

Of the 917 invited participants, 316 responded to the survey and completed at least three- 
quarters of the survey, providing a 34.5% response rate. This response suggests the potential 
for cautious generalizing to the national level and could be strengthened with greater response 
in future administrations. Instances of overrepresentation within the strata were not noted. 

Institutional Role 

The Survey of Assessment Cultures first section collects data on the roles respondents hold on 
their campus. Participants were asked to respond to the question: “Does your institution 
employ a Chief Academic Officer?” The survey defines the role of a Chief Assessment 
Officer as “an individual for whom assessment is their sole responsibility on their campus” 
and asks participants to indicate if they, another individual, many individuals, or no 
individuals on their campus fulfill this role. Nearly a third of respondents (31.0% or 98 
participants) indicated they were the Chief Assessment Officer for their campus while over 
half (54.3% or 172 participants) indicated their campus delivers assessment through many 
practitioners. Participants could describe their role on campus as a Chief Assessment Officer 
(31.0% or 98 participants), identify another colleague as the Chief Assessment Officer (10.1% 
or 32 participants), indicate their campus employs many individuals to lead assessment 
(54.3% or 172 participants), indicate no one at their institution holds that role (4.4% or 14 
participants), or that they were unsure if anyone held such a position (0.6% or 2 participants). 
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Institutional Resources 


Maki (2010) suggests one of the most important roles an institution can have in place is to 
formally task an individual or individuals with the responsibility of meeting with faculty when 
questions about assessment arise. To this end, the Survey asks participants if they are the 
person formally tasked with meeting with faculty to support this need. Two hundred and 
thirteen participants (67.4%) indicated they were the primary person with whom faculty 
should meet to seek assistance in assessment activities. In contrast, 76 participants (24.1%) 
indicated they were not the primary person tasked with this responsibility, 20 participants 
(6.3%) indicated no one held this responsibility on their campus, and 7 participants (2.2%) 
indicated they were not sure if anyone on their campus held this responsibility. 

Purpose of Assessment 

Assessment can be done for a variety of reasons and the intent with which it is done gives 
assessment practice a perspective driving its practice in both apparent and hidden ways. To 
explore this phenomenon, the Survey asks participants to complete a sentence describing the 

reason assessment is done on their campus: [ “_ is the primary reason 

assessment is done on our campus.” ]. Respondents had to complete the sentence using only 
one of the following randomly-ordered selections derived from Maki’s (2010) purposes for 
assessment: Access to financial resources, Accountability, Accreditation, Compliance with 
governmental mandates, Improving student learning, or Tradition. Subsequent open-ended 
questions asked participants to share additional reasons assessment is done on their campus 
and expound upon their reasoning for answering this question as they did. Table 1 offers 
frequency and percentage statistics for this question. 

Table 1 

Reason Assessment Done 


Reason 

Count 

Percentage 

Improving student learning 

151 

49.0 

Accreditation 

125 

40.6 

Accountability 

26 

8.4 

Compliance with government mandates 

3 

1.2 

Tradition 

2 

0.8 

Total 

307 

100 


Support, Resistance, and Indifference to Assessment 

Respondents were asked to rank a variety of campus leaders regarding their supportiveness, 
resistance, or indifference/unawareness to assessment. A seven-point Likert-type scale was 
developed ranging from “Highly Resistant” (1) to “Highly Supportive” (7). Respondents 
could also indicate if they perceived specific campus leaders to be “Indifferent/Unaware of 
assessment” (0) and if they held the position being ranked. In the case of the latter, 
participants’ responses on their own performance are not included in these results. Indifferent 
or unaware rankings were assigned a value of 0, reflecting the qualitative nature of this label. 
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Table 2 depicts data as percentages of respondents that are the most supportive, resistant, or 
indifferent to assessment. Responses are collapsed down across rankings to “Supportive” 
“Resistant,” and “Unaware/Indifferent” categories. The campus president, provost, and 
student affairs/services staff command the greatest percentage of “Supportive” rankings 
(91.6%, 90.6%, and 88.5%, respectively). Faculty and faculty/academic senate leadership are 
the only groups to obtain relatively large percentages of rankings in the “Resistant” categories 
(22.9% and 17.4%, respectively), though even these percentages can be considered minimal. 

Table 2 

Aggregated Support, Resistance, or Indifference/Unawareness to Assessment 


Group 

% Supportive 

% Resistant 

% 

Indiff./Unaware 

Board of Trustee Members 

69.5 

0.0 

30.5 

President 

91.6 

2.2 

6.2 

Provost 

90.6 

4.2 

5.2 

Faculty 

75.8 

22.9 

1.3 

Student Affairs 

88.5 

5.8 

5.8 

Administrators 

Faculty Senate Leaders 

78.9 

17.0 

4.0 

Fundraising/Devel. Officers 

53.4 

3.7 

42.9 

Alumni groups 

29.0 

1.4 

69.5 

Academic Advisors 

73.2 

7.3 

19.6 

Student Government Leaders 

49.3 

1.4 

49.3 

Overall 

70.1 

6.7 

23.1 


DISCUSSION 

These descriptive findings hold promise for future explorations and dialogue. First, the fact 
that over half of participants indicated their institution delivered assessment through many 
leaders suggests the need to take into account this multi-leader form of administration. 
America’s approach to assessment is largely by a distributed model. While this offers 
opportunities to explore connections to dialogical and participatory theories it also challenges 
traditional leadership theories. Seemingly, only a third of participants possess the fonnal 
authority of being a Chief Assessment Officer. Half of the respondents possess a different, 
less fonnal form of authority; one requiring unique skills of negotiation, finesse, and the 
ability to balance shared responsibilities and academic governance. This is not to say Chief 
Assessment Officers or do not require similar skills. Indeed, their role also requires them to 
negotiate more frequently than they mandate. Nonetheless, this finding suggests the need to 
further explore theories of leadership, organization, and administration that take into account 
the shared nature of assessment and the unique contexts of being one leader among many. 

Data regarding the reason assessment is done on a campus also offer opportunities for 
further dialogue. Almost half (49.0%) of the respondents indicate improvement of student 
learning is the reason assessment is done. For some, this could be a point worth celebrating as 
it supports the general logic and noble intentions outlined in assessment scholarship. For 
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others, the fact that less than half of the respondents indicated improvement of student 
learning as the primary reason assessment is done may not be “enough.” For every 
respondent focusing on student learning, there is another whose attention is turned to 
“accountability” or “accreditation” (40.6% and 8.4%, respectively for a total of 48.9%) as the 
primary reason assessment is done. 

Assessment leaders should strive to maintain appropriate focus on student learning 
while responding to the pragmatic, day-to day pressures of assessment, accountability, and 
accreditation efforts. Wolff (2005) advocated the explanation of accreditation as “a bulwark 
for quality in an environment where institutions are buffeted by state priorities to increase 
institutional access, improve graduation rates, and operate with less financial support” (p. 78). 
Educational administrators must be prepared and capable of underscoring that accreditation 
exists to support student learning. In an increasingly complex relationship between 
institutions and governing bodies, faculty, and administrators, campus leadership must refine 
and reiterate messages about the importance of student learning in institutional operations and 
accreditation. 

The findings pertaining to those constituents who are most supportive, resistant, or 
indifferent to assessment may aid campus leaders in support the advancement of assessment 
messages and practices. Empowering supportive campus leaders to engage indifferent or 
unaware campus leaders may prove beneficial. Partnerships between supportive and resistant 
or indifferent groups may also be useful in educating more campus leaders about the benefits 
and processes of assessment. Furthermore, traditional narratives espouse high levels of 
faculty resistance to assessment (Driscoll, de Noriega, & Ramaley, 2006) or illustrate the 
notion of academic gamesmanship (Astin & antonio [sic ], 2012) faculty employ to diminish 
assessment’s prominence or refute its purpose. Findings from the current study reveal a more 
positive belief in faculty than may be traditionally assumed. Although assessment 
administrators perceive faculty and academic senate leaders as more resistant than other 
groups, faculty and academic senate leaders could hardly be described as “highly resistant” to 
assessment based upon these data. Faculty members have daily contact with students and are 
vital collaborators in an effective culture of assessment focused on improving student 
learning. These data generally support the notion that faculty members are as supportive of 
assessment and educational administrators must work to deliver these positive findings 
throughout their campus constituencies. Doing so may resolve many traditional barriers to 
faculty participation and advocacy for assessment in higher education. 

Lastly, these findings may be of importance to faculty in graduate level educational 
administration programs. Graduate students preparing entry into higher education 
administration must carry with them the ability to engage in respectful generative dialogue 
about meaningful aspects of teaching, learning, quality, and higher education administration 
(Fuller, 2012). Administrators can do much to gain legitimacy in the eyes of professors and 
staff if they can construct reasonable plans for improvement using sound empirical evidence, 
a skill often honed in graduate school. The data provide insights into the contexts for which 
graduate students must be prepared as future educational leaders. 

CONCLUSION 

Scholars and researchers exploring the art and science of assessment may find these results 
useful in outlining plans for future research or crafting theories about assessment cultures and 
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practices. However, the true significance of these data may be in their relevance to the 
scholarship on assessment and the new questions they pose about assessment cultures. 
Readers may see any number of interesting findings in the data and further research using the 
Survey of Assessment Culture is necessary and has already begun. If these data offer insights 
or intriguing possibilities for future research, they have made a meaningful contribution to this 
complex line of emerging scholarship. 
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